{
 "nbformat": 4,
 "nbformat_minor": 0,
 "metadata": {
  "colab": {
   "name": "MMAction2 Tutorial.ipynb",
   "provenance": [],
   "collapsed_sections": [],
   "toc_visible": true,
   "include_colab_link": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  },
  "accelerator": "GPU"
 },
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "view-in-github",
    "colab_type": "text"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/open-mmlab/mmaction2/blob/master/demo/mmaction2_tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VcjSRFELVbNk"
   },
   "source": [
    "# MMAction2 Tutorial\n",
    "\n",
    "Welcome to MMAction2! This is the official colab tutorial for using MMAction2. In this tutorial, you will learn\n",
    "- Perform inference with a MMAction2 recognizer.\n",
    "- Train a new recognizer with a new dataset.\n",
    "\n",
    "Let's start!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7LqHGkGEVqpm"
   },
   "source": [
    "## Install MMAction2"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "Bf8PpPXtVvmg",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "2c685a33-474b-4e71-8f98-c2533c66095e"
   },
   "source": [
    "# Check nvcc version\n",
    "!nvcc -V\n",
    "# Check GCC version\n",
    "!gcc --version"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "nvcc: NVIDIA (R) Cuda compiler driver\n",
      "Copyright (c) 2005-2019 NVIDIA Corporation\n",
      "Built on Sun_Jul_28_19:07:16_PDT_2019\n",
      "Cuda compilation tools, release 10.1, V10.1.243\n",
      "gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0\n",
      "Copyright (C) 2017 Free Software Foundation, Inc.\n",
      "This is free software; see the source for copying conditions.  There is NO\n",
      "warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n",
      "\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "5PAJ4ArzV5Ry",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "e48dbf61-fae0-431c-e964-04c7caaee4bc"
   },
   "source": [
    "# install dependencies: (use cu101 because colab has CUDA 10.1)\n",
    "!pip install -U torch==1.5.1+cu101 torchvision==0.6.1+cu101 -f https://download.pytorch.org/whl/torch_stable.html\n",
    "\n",
    "# install mmcv-full thus we could use CUDA operators\n",
    "!pip install mmcv-full==latest+torch1.5.0+cu101 -f https://download.openmmlab.com/mmcv/dist/index.html\n",
    "\n",
    "# Install mmaction2\n",
    "!rm -rf mmaction2\n",
    "!git clone https://github.com/open-mmlab/mmaction2.git\n",
    "%cd mmaction2\n",
    "\n",
    "!pip install -e .\n",
    "\n",
    "# Install some optional requirements\n",
    "!pip install -r requirements/optional.txt"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
      "Requirement already up-to-date: torch==1.5.1+cu101 in /usr/local/lib/python3.6/dist-packages (1.5.1+cu101)\n",
      "Requirement already up-to-date: torchvision==0.6.1+cu101 in /usr/local/lib/python3.6/dist-packages (0.6.1+cu101)\n",
      "Requirement already satisfied, skipping upgrade: future in /usr/local/lib/python3.6/dist-packages (from torch==1.5.1+cu101) (0.16.0)\n",
      "Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from torch==1.5.1+cu101) (1.18.5)\n",
      "Requirement already satisfied, skipping upgrade: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision==0.6.1+cu101) (7.0.0)\n",
      "Looking in links: https://download.openmmlab.com/mmcv/dist/index.html\n",
      "Collecting mmcv-full==latest+torch1.5.0+cu101\n",
      "  Using cached https://download.openmmlab.com/mmcv/dist/latest/torch1.5.0/cu101/mmcv_full-latest%2Btorch1.5.0%2Bcu101-cp36-cp36m-manylinux1_x86_64.whl\n",
      "Requirement already satisfied: Pillow in /usr/local/lib/python3.6/dist-packages (from mmcv-full==latest+torch1.5.0+cu101) (7.0.0)\n",
      "Requirement already satisfied: addict in /usr/local/lib/python3.6/dist-packages (from mmcv-full==latest+torch1.5.0+cu101) (2.3.0)\n",
      "Requirement already satisfied: yapf in /usr/local/lib/python3.6/dist-packages (from mmcv-full==latest+torch1.5.0+cu101) (0.30.0)\n",
      "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from mmcv-full==latest+torch1.5.0+cu101) (1.18.5)\n",
      "Requirement already satisfied: opencv-python>=3 in /usr/local/lib/python3.6/dist-packages (from mmcv-full==latest+torch1.5.0+cu101) (4.1.2.30)\n",
      "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from mmcv-full==latest+torch1.5.0+cu101) (3.13)\n",
      "Installing collected packages: mmcv-full\n",
      "  Found existing installation: mmcv-full 1.2.0\n",
      "    Uninstalling mmcv-full-1.2.0:\n",
      "      Successfully uninstalled mmcv-full-1.2.0\n",
      "Successfully installed mmcv-full-1.2.0\n",
      "Cloning into 'mmaction2'...\n",
      "remote: Enumerating objects: 8, done.\u001b[K\n",
      "remote: Counting objects: 100% (8/8), done.\u001b[K\n",
      "remote: Compressing objects: 100% (8/8), done.\u001b[K\n",
      "remote: Total 7596 (delta 0), reused 1 (delta 0), pack-reused 7588\u001b[K\n",
      "Receiving objects: 100% (7596/7596), 33.53 MiB | 37.44 MiB/s, done.\n",
      "Resolving deltas: 100% (5329/5329), done.\n",
      "/content/mmaction2\n",
      "Obtaining file:///content/mmaction2\n",
      "Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from mmaction2==0.8.0) (3.2.2)\n",
      "Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from mmaction2==0.8.0) (1.18.5)\n",
      "Requirement already satisfied: opencv-contrib-python in /usr/local/lib/python3.6/dist-packages (from mmaction2==0.8.0) (4.1.2.30)\n",
      "Requirement already satisfied: Pillow in /usr/local/lib/python3.6/dist-packages (from mmaction2==0.8.0) (7.0.0)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->mmaction2==0.8.0) (1.3.1)\n",
      "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->mmaction2==0.8.0) (2.8.1)\n",
      "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->mmaction2==0.8.0) (2.4.7)\n",
      "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->mmaction2==0.8.0) (0.10.0)\n",
      "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.1->matplotlib->mmaction2==0.8.0) (1.15.0)\n",
      "Installing collected packages: mmaction2\n",
      "  Running setup.py develop for mmaction2\n",
      "Successfully installed mmaction2\n",
      "Requirement already satisfied: av in /usr/local/lib/python3.6/dist-packages (from -r requirements/optional.txt (line 1)) (8.0.2)\n",
      "Requirement already satisfied: decord>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from -r requirements/optional.txt (line 2)) (0.4.2)\n",
      "Requirement already satisfied: moviepy in /usr/local/lib/python3.6/dist-packages (from -r requirements/optional.txt (line 3)) (0.2.3.5)\n",
      "Requirement already satisfied: onnx in /usr/local/lib/python3.6/dist-packages (from -r requirements/optional.txt (line 4)) (1.8.0)\n",
      "Requirement already satisfied: onnxruntime in /usr/local/lib/python3.6/dist-packages (from -r requirements/optional.txt (line 5)) (1.5.2)\n",
      "Requirement already satisfied: PyTurboJPEG in /usr/local/lib/python3.6/dist-packages (from -r requirements/optional.txt (line 6)) (1.4.1)\n",
      "Requirement already satisfied: numpy>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from decord>=0.4.1->-r requirements/optional.txt (line 2)) (1.18.5)\n",
      "Requirement already satisfied: tqdm<5.0,>=4.11.2 in /usr/local/lib/python3.6/dist-packages (from moviepy->-r requirements/optional.txt (line 3)) (4.41.1)\n",
      "Requirement already satisfied: decorator<5.0,>=4.0.2 in /usr/local/lib/python3.6/dist-packages (from moviepy->-r requirements/optional.txt (line 3)) (4.4.2)\n",
      "Requirement already satisfied: imageio<3.0,>=2.1.2 in /usr/local/lib/python3.6/dist-packages (from moviepy->-r requirements/optional.txt (line 3)) (2.4.1)\n",
      "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from onnx->-r requirements/optional.txt (line 4)) (1.15.0)\n",
      "Requirement already satisfied: protobuf in /usr/local/lib/python3.6/dist-packages (from onnx->-r requirements/optional.txt (line 4)) (3.12.4)\n",
      "Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.6/dist-packages (from onnx->-r requirements/optional.txt (line 4)) (3.7.4.3)\n",
      "Requirement already satisfied: pillow in /usr/local/lib/python3.6/dist-packages (from imageio<3.0,>=2.1.2->moviepy->-r requirements/optional.txt (line 3)) (7.0.0)\n",
      "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf->onnx->-r requirements/optional.txt (line 4)) (50.3.2)\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "No_zZAFpWC-a",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "1d425eea-d44e-434a-991c-01eb15abaab2"
   },
   "source": [
    "# Check Pytorch installation\n",
    "import torch, torchvision\n",
    "print(torch.__version__, torch.cuda.is_available())\n",
    "\n",
    "# Check MMAction2 installation\n",
    "import mmaction\n",
    "print(mmaction.__version__)\n",
    "\n",
    "# Check MMCV installation\n",
    "from mmcv.ops import get_compiling_cuda_version, get_compiler_version\n",
    "print(get_compiling_cuda_version())\n",
    "print(get_compiler_version())"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "1.5.1+cu101 True\n",
      "0.8.0\n",
      "10.1\n",
      "GCC 7.3\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pXf7oV5DWdab"
   },
   "source": [
    "## Perform inference with a MMAction2 recognizer\n",
    "MMAction2 already provides high level APIs to do inference and training."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "64CW6d_AaT-Q",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "3b284fd8-4ee7-4a34-90d7-5023cd123a04"
   },
   "source": [
    "!mkdir checkpoints\n",
    "!wget -c https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth \\\n",
    "      -O checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "--2020-11-20 09:38:27--  https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth\n",
      "Resolving download.openmmlab.com (download.openmmlab.com)... 47.252.96.35\n",
      "Connecting to download.openmmlab.com (download.openmmlab.com)|47.252.96.35|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 97579339 (93M) [application/octet-stream]\n",
      "Saving to: ‘checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth’\n",
      "\n",
      "checkpoints/tsn_r50 100%[===================>]  93.06M  8.48MB/s    in 11s     \n",
      "\n",
      "2020-11-20 09:38:40 (8.49 MB/s) - ‘checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth’ saved [97579339/97579339]\n",
      "\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "HNZB7NoSabzj"
   },
   "source": [
    "from mmaction.apis import inference_recognizer, init_recognizer\n",
    "\n",
    "# Choose to use a config and initialize the recognizer\n",
    "config = 'configs/recognition/tsn/tsn_r50_video_inference_1x1x3_100e_kinetics400_rgb.py'\n",
    "# Setup a checkpoint file to load\n",
    "checkpoint = 'checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'\n",
    "# Initialize the recognizer\n",
    "model = init_recognizer(config, checkpoint, device='cuda:0')"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "rEMsBnpHapAn"
   },
   "source": [
    "# Use the recognizer to do inference\n",
    "video = 'demo/demo.mp4'\n",
    "label = 'demo/label_map_k400.txt'\n",
    "results = inference_recognizer(model, video, label)"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "NIyJXqfWathq",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "a5ac09b9-d7df-4430-a89e-28526b76f3be"
   },
   "source": [
    "# Let's show the results\n",
    "for result in results:\n",
    "    print(f'{result[0]}: ', result[1])"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "arm wrestling:  29.616438\n",
      "rock scissors paper:  10.754841\n",
      "shaking hands:  9.908401\n",
      "clapping:  9.189913\n",
      "massaging feet:  8.305306\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QuZG8kZ2fJ5d"
   },
   "source": [
    "## Train a recognizer on customized dataset\n",
    "\n",
    "To train a new recognizer, there are usually three things to do:\n",
    "1. Support a new dataset\n",
    "2. Modify the config\n",
    "3. Train a new recognizer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "neEFyxChfgiJ"
   },
   "source": [
    "### Support a new dataset\n",
    "\n",
    "In this tutorial, we gives an example to convert the data into the format of existing datasets. Other methods and more advanced usages can be found in the [doc](/docs/tutorials/new_dataset.md)\n",
    "\n",
    "Firstly, let's download a tiny dataset obtained from [Kinetics-400](https://deepmind.com/research/open-source/open-source-datasets/kinetics/). We select 30 videos with their labels as train dataset and 10 videos with their labels as test dataset."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "gjsUj9JzgUlJ",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "2bc7af2f-0068-430c-d8f2-f85413466b85"
   },
   "source": [
    "# download, decompress the data\n",
    "!rm kinetics400_tiny.zip*\n",
    "!rm -rf kinetics400_tiny\n",
    "!wget https://download.openmmlab.com/mmaction/kinetics400_tiny.zip\n",
    "!unzip kinetics400_tiny.zip > /dev/null"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "rm: cannot remove 'kinetics400_tiny.zip*': No such file or directory\n",
      "--2020-11-20 09:38:48--  https://download.openmmlab.com/mmaction/kinetics400_tiny.zip\n",
      "Resolving download.openmmlab.com (download.openmmlab.com)... 47.252.96.35\n",
      "Connecting to download.openmmlab.com (download.openmmlab.com)|47.252.96.35|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 18308682 (17M) [application/zip]\n",
      "Saving to: ‘kinetics400_tiny.zip’\n",
      "\n",
      "kinetics400_tiny.zi 100%[===================>]  17.46M  8.59MB/s    in 2.0s    \n",
      "\n",
      "2020-11-20 09:38:51 (8.59 MB/s) - ‘kinetics400_tiny.zip’ saved [18308682/18308682]\n",
      "\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "AbZ-o7V6hNw4",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "a8ad33e5-7d29-47ca-f14e-bb356ea096eb"
   },
   "source": [
    "# Check the directory structure of the tiny data\n",
    "\n",
    "# Install tree first\n",
    "!apt-get -q install tree\n",
    "!tree kinetics400_tiny"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "Reading package lists...\n",
      "Building dependency tree...\n",
      "Reading state information...\n",
      "tree is already the newest version (1.7.0-5).\n",
      "0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded.\n",
      "kinetics400_tiny\n",
      "├── kinetics_tiny_train_video.txt\n",
      "├── kinetics_tiny_val_video.txt\n",
      "├── train\n",
      "│   ├── 27_CSXByd3s.mp4\n",
      "│   ├── 34XczvTaRiI.mp4\n",
      "│   ├── A-wiliK50Zw.mp4\n",
      "│   ├── D32_1gwq35E.mp4\n",
      "│   ├── D92m0HsHjcQ.mp4\n",
      "│   ├── DbX8mPslRXg.mp4\n",
      "│   ├── FMlSTTpN3VY.mp4\n",
      "│   ├── h10B9SVE-nk.mp4\n",
      "│   ├── h2YqqUhnR34.mp4\n",
      "│   ├── iRuyZSKhHRg.mp4\n",
      "│   ├── IyfILH9lBRo.mp4\n",
      "│   ├── kFC3KY2bOP8.mp4\n",
      "│   ├── LvcFDgCAXQs.mp4\n",
      "│   ├── O46YA8tI530.mp4\n",
      "│   ├── oMrZaozOvdQ.mp4\n",
      "│   ├── oXy-e_P_cAI.mp4\n",
      "│   ├── P5M-hAts7MQ.mp4\n",
      "│   ├── phDqGd0NKoo.mp4\n",
      "│   ├── PnOe3GZRVX8.mp4\n",
      "│   ├── R8HXQkdgKWA.mp4\n",
      "│   ├── RqnKtCEoEcA.mp4\n",
      "│   ├── soEcZZsBmDs.mp4\n",
      "│   ├── TkkZPZHbAKA.mp4\n",
      "│   ├── T_TMNGzVrDk.mp4\n",
      "│   ├── WaS0qwP46Us.mp4\n",
      "│   ├── Wh_YPQdH1Zg.mp4\n",
      "│   ├── WWP5HZJsg-o.mp4\n",
      "│   ├── xGY2dP0YUjA.mp4\n",
      "│   ├── yLC9CtWU5ws.mp4\n",
      "│   └── ZQV4U2KQ370.mp4\n",
      "└── val\n",
      "    ├── 0pVGiAU6XEA.mp4\n",
      "    ├── AQrbRSnRt8M.mp4\n",
      "    ├── b6Q_b7vgc7Q.mp4\n",
      "    ├── ddvJ6-faICE.mp4\n",
      "    ├── IcLztCtvhb8.mp4\n",
      "    ├── ik4BW3-SCts.mp4\n",
      "    ├── jqRrH30V0k4.mp4\n",
      "    ├── SU_x2LQqSLs.mp4\n",
      "    ├── u4Rm6srmIS8.mp4\n",
      "    └── y5Iu7XkTqV0.mp4\n",
      "\n",
      "2 directories, 42 files\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "fTdi6dI0hY3g",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "a78557e3-2a82-4c5b-a292-017660cc5b5e"
   },
   "source": [
    "# After downloading the data, we need to check the annotation format\n",
    "!cat kinetics400_tiny/kinetics_tiny_train_video.txt"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "D32_1gwq35E.mp4 0\n",
      "iRuyZSKhHRg.mp4 1\n",
      "oXy-e_P_cAI.mp4 0\n",
      "34XczvTaRiI.mp4 1\n",
      "h2YqqUhnR34.mp4 0\n",
      "O46YA8tI530.mp4 0\n",
      "kFC3KY2bOP8.mp4 1\n",
      "WWP5HZJsg-o.mp4 1\n",
      "phDqGd0NKoo.mp4 1\n",
      "yLC9CtWU5ws.mp4 0\n",
      "27_CSXByd3s.mp4 1\n",
      "IyfILH9lBRo.mp4 1\n",
      "T_TMNGzVrDk.mp4 1\n",
      "TkkZPZHbAKA.mp4 0\n",
      "PnOe3GZRVX8.mp4 1\n",
      "soEcZZsBmDs.mp4 1\n",
      "FMlSTTpN3VY.mp4 1\n",
      "WaS0qwP46Us.mp4 0\n",
      "A-wiliK50Zw.mp4 1\n",
      "oMrZaozOvdQ.mp4 1\n",
      "ZQV4U2KQ370.mp4 0\n",
      "DbX8mPslRXg.mp4 1\n",
      "h10B9SVE-nk.mp4 1\n",
      "P5M-hAts7MQ.mp4 0\n",
      "R8HXQkdgKWA.mp4 0\n",
      "D92m0HsHjcQ.mp4 0\n",
      "RqnKtCEoEcA.mp4 0\n",
      "LvcFDgCAXQs.mp4 0\n",
      "xGY2dP0YUjA.mp4 0\n",
      "Wh_YPQdH1Zg.mp4 0\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0bq0mxmEi29H"
   },
   "source": [
    "According to the format defined in [`VideoDataset`](./datasets/video_dataset.py), each line indicates a sample video with the filepath and label, which are split with a whitespace."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ht_DGJA9jQar"
   },
   "source": [
    "### Modify the config\n",
    "\n",
    "In the next step, we need to modify the config for the training.\n",
    "To accelerate the process, we finetune a recognizer using a pre-trained recognizer."
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "LjCcmCKOjktc"
   },
   "source": [
    "from mmcv import Config\n",
    "cfg = Config.fromfile('./configs/recognition/tsn/tsn_r50_video_1x1x8_100e_kinetics400_rgb.py')"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tc8YhFFGjp3e"
   },
   "source": [
    "Given a config that trains a TSN model on kinetics400-full dataset, we need to modify some values to use it for training TSN on Kinetics400-tiny dataset.\n"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "tlhu9byjjt-K",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "13d1bb01-f351-48c0-9efb-0bdf2c125d29"
   },
   "source": [
    "from mmcv.runner import set_random_seed\n",
    "\n",
    "# Modify dataset type and path\n",
    "cfg.dataset_type = 'VideoDataset'\n",
    "cfg.data_root = 'kinetics400_tiny/train/'\n",
    "cfg.data_root_val = 'kinetics400_tiny/val/'\n",
    "cfg.ann_file_train = 'kinetics400_tiny/kinetics_tiny_train_video.txt'\n",
    "cfg.ann_file_val = 'kinetics400_tiny/kinetics_tiny_val_video.txt'\n",
    "cfg.ann_file_test = 'kinetics400_tiny/kinetics_tiny_val_video.txt'\n",
    "\n",
    "cfg.data.test.type = 'VideoDataset'\n",
    "cfg.data.test.ann_file = 'kinetics400_tiny/kinetics_tiny_val_video.txt'\n",
    "cfg.data.test.data_prefix = 'kinetics400_tiny/val/'\n",
    "\n",
    "cfg.data.train.type = 'VideoDataset'\n",
    "cfg.data.train.ann_file = 'kinetics400_tiny/kinetics_tiny_train_video.txt'\n",
    "cfg.data.train.data_prefix = 'kinetics400_tiny/train/'\n",
    "\n",
    "cfg.data.val.type = 'VideoDataset'\n",
    "cfg.data.val.ann_file = 'kinetics400_tiny/kinetics_tiny_val_video.txt'\n",
    "cfg.data.val.data_prefix = 'kinetics400_tiny/val/'\n",
    "\n",
    "# The flag is used to determine whether it is omnisource training\n",
    "cfg.setdefault('omnisource', False)\n",
    "# Modify num classes of the model in cls_head\n",
    "cfg.model.cls_head.num_classes = 2\n",
    "# We can use the pre-trained TSN model\n",
    "cfg.load_from = './checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'\n",
    "\n",
    "# Set up working dir to save files and logs.\n",
    "cfg.work_dir = './tutorial_exps'\n",
    "\n",
    "# The original learning rate (LR) is set for 8-GPU training.\n",
    "# We divide it by 8 since we only use one GPU.\n",
    "cfg.data.videos_per_gpu = cfg.data.videos_per_gpu // 16\n",
    "cfg.optimizer.lr = cfg.optimizer.lr / 8 / 16\n",
    "cfg.total_epochs = 30\n",
    "\n",
    "# We can set the checkpoint saving interval to reduce the storage cost\n",
    "cfg.checkpoint_config.interval = 10\n",
    "# We can set the log print interval to reduce the the times of printing log\n",
    "cfg.log_config.interval = 5\n",
    "\n",
    "# Set seed thus the results are more reproducible\n",
    "cfg.seed = 0\n",
    "set_random_seed(0, deterministic=False)\n",
    "cfg.gpu_ids = range(1)\n",
    "\n",
    "\n",
    "# We can initialize the logger for training and have a look\n",
    "# at the final config used for training\n",
    "print(f'Config:\\n{cfg.pretty_text}')\n"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "Config:\n",
      "model = dict(\n",
      "    type='Recognizer2D',\n",
      "    backbone=dict(\n",
      "        type='ResNet',\n",
      "        pretrained='torchvision://resnet50',\n",
      "        depth=50,\n",
      "        norm_eval=False),\n",
      "    cls_head=dict(\n",
      "        type='TSNHead',\n",
      "        num_classes=2,\n",
      "        in_channels=2048,\n",
      "        spatial_type='avg',\n",
      "        consensus=dict(type='AvgConsensus', dim=1),\n",
      "        dropout_ratio=0.4,\n",
      "        init_std=0.01))\n",
      "train_cfg = None\n",
      "test_cfg = dict(average_clips=None)\n",
      "dataset_type = 'VideoDataset'\n",
      "data_root = 'kinetics400_tiny/train/'\n",
      "data_root_val = 'kinetics400_tiny/val/'\n",
      "ann_file_train = 'kinetics400_tiny/kinetics_tiny_train_video.txt'\n",
      "ann_file_val = 'kinetics400_tiny/kinetics_tiny_val_video.txt'\n",
      "ann_file_test = 'kinetics400_tiny/kinetics_tiny_val_video.txt'\n",
      "img_norm_cfg = dict(\n",
      "    mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_bgr=False)\n",
      "train_pipeline = [\n",
      "    dict(type='DecordInit'),\n",
      "    dict(type='SampleFrames', clip_len=1, frame_interval=1, num_clips=8),\n",
      "    dict(type='DecordDecode'),\n",
      "    dict(\n",
      "        type='MultiScaleCrop',\n",
      "        input_size=224,\n",
      "        scales=(1, 0.875, 0.75, 0.66),\n",
      "        random_crop=False,\n",
      "        max_wh_scale_gap=1),\n",
      "    dict(type='Resize', scale=(224, 224), keep_ratio=False),\n",
      "    dict(type='Flip', flip_ratio=0.5),\n",
      "    dict(\n",
      "        type='Normalize',\n",
      "        mean=[123.675, 116.28, 103.53],\n",
      "        std=[58.395, 57.12, 57.375],\n",
      "        to_bgr=False),\n",
      "    dict(type='FormatShape', input_format='NCHW'),\n",
      "    dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),\n",
      "    dict(type='ToTensor', keys=['imgs', 'label'])\n",
      "]\n",
      "val_pipeline = [\n",
      "    dict(type='DecordInit'),\n",
      "    dict(\n",
      "        type='SampleFrames',\n",
      "        clip_len=1,\n",
      "        frame_interval=1,\n",
      "        num_clips=8,\n",
      "        test_mode=True),\n",
      "    dict(type='DecordDecode'),\n",
      "    dict(type='Resize', scale=(-1, 256)),\n",
      "    dict(type='CenterCrop', crop_size=224),\n",
      "    dict(type='Flip', flip_ratio=0),\n",
      "    dict(\n",
      "        type='Normalize',\n",
      "        mean=[123.675, 116.28, 103.53],\n",
      "        std=[58.395, 57.12, 57.375],\n",
      "        to_bgr=False),\n",
      "    dict(type='FormatShape', input_format='NCHW'),\n",
      "    dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),\n",
      "    dict(type='ToTensor', keys=['imgs'])\n",
      "]\n",
      "test_pipeline = [\n",
      "    dict(type='DecordInit'),\n",
      "    dict(\n",
      "        type='SampleFrames',\n",
      "        clip_len=1,\n",
      "        frame_interval=1,\n",
      "        num_clips=25,\n",
      "        test_mode=True),\n",
      "    dict(type='DecordDecode'),\n",
      "    dict(type='Resize', scale=(-1, 256)),\n",
      "    dict(type='ThreeCrop', crop_size=256),\n",
      "    dict(type='Flip', flip_ratio=0),\n",
      "    dict(\n",
      "        type='Normalize',\n",
      "        mean=[123.675, 116.28, 103.53],\n",
      "        std=[58.395, 57.12, 57.375],\n",
      "        to_bgr=False),\n",
      "    dict(type='FormatShape', input_format='NCHW'),\n",
      "    dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),\n",
      "    dict(type='ToTensor', keys=['imgs'])\n",
      "]\n",
      "data = dict(\n",
      "    videos_per_gpu=2,\n",
      "    workers_per_gpu=4,\n",
      "    train=dict(\n",
      "        type='VideoDataset',\n",
      "        ann_file='kinetics400_tiny/kinetics_tiny_train_video.txt',\n",
      "        data_prefix='kinetics400_tiny/train/',\n",
      "        pipeline=[\n",
      "            dict(type='DecordInit'),\n",
      "            dict(\n",
      "                type='SampleFrames', clip_len=1, frame_interval=1,\n",
      "                num_clips=8),\n",
      "            dict(type='DecordDecode'),\n",
      "            dict(\n",
      "                type='MultiScaleCrop',\n",
      "                input_size=224,\n",
      "                scales=(1, 0.875, 0.75, 0.66),\n",
      "                random_crop=False,\n",
      "                max_wh_scale_gap=1),\n",
      "            dict(type='Resize', scale=(224, 224), keep_ratio=False),\n",
      "            dict(type='Flip', flip_ratio=0.5),\n",
      "            dict(\n",
      "                type='Normalize',\n",
      "                mean=[123.675, 116.28, 103.53],\n",
      "                std=[58.395, 57.12, 57.375],\n",
      "                to_bgr=False),\n",
      "            dict(type='FormatShape', input_format='NCHW'),\n",
      "            dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),\n",
      "            dict(type='ToTensor', keys=['imgs', 'label'])\n",
      "        ]),\n",
      "    val=dict(\n",
      "        type='VideoDataset',\n",
      "        ann_file='kinetics400_tiny/kinetics_tiny_val_video.txt',\n",
      "        data_prefix='kinetics400_tiny/val/',\n",
      "        pipeline=[\n",
      "            dict(type='DecordInit'),\n",
      "            dict(\n",
      "                type='SampleFrames',\n",
      "                clip_len=1,\n",
      "                frame_interval=1,\n",
      "                num_clips=8,\n",
      "                test_mode=True),\n",
      "            dict(type='DecordDecode'),\n",
      "            dict(type='Resize', scale=(-1, 256)),\n",
      "            dict(type='CenterCrop', crop_size=224),\n",
      "            dict(type='Flip', flip_ratio=0),\n",
      "            dict(\n",
      "                type='Normalize',\n",
      "                mean=[123.675, 116.28, 103.53],\n",
      "                std=[58.395, 57.12, 57.375],\n",
      "                to_bgr=False),\n",
      "            dict(type='FormatShape', input_format='NCHW'),\n",
      "            dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),\n",
      "            dict(type='ToTensor', keys=['imgs'])\n",
      "        ]),\n",
      "    test=dict(\n",
      "        type='VideoDataset',\n",
      "        ann_file='kinetics400_tiny/kinetics_tiny_val_video.txt',\n",
      "        data_prefix='kinetics400_tiny/val/',\n",
      "        pipeline=[\n",
      "            dict(type='DecordInit'),\n",
      "            dict(\n",
      "                type='SampleFrames',\n",
      "                clip_len=1,\n",
      "                frame_interval=1,\n",
      "                num_clips=25,\n",
      "                test_mode=True),\n",
      "            dict(type='DecordDecode'),\n",
      "            dict(type='Resize', scale=(-1, 256)),\n",
      "            dict(type='ThreeCrop', crop_size=256),\n",
      "            dict(type='Flip', flip_ratio=0),\n",
      "            dict(\n",
      "                type='Normalize',\n",
      "                mean=[123.675, 116.28, 103.53],\n",
      "                std=[58.395, 57.12, 57.375],\n",
      "                to_bgr=False),\n",
      "            dict(type='FormatShape', input_format='NCHW'),\n",
      "            dict(type='Collect', keys=['imgs', 'label'], meta_keys=[]),\n",
      "            dict(type='ToTensor', keys=['imgs'])\n",
      "        ]))\n",
      "optimizer = dict(type='SGD', lr=7.8125e-05, momentum=0.9, weight_decay=0.0001)\n",
      "optimizer_config = dict(grad_clip=dict(max_norm=40, norm_type=2))\n",
      "lr_config = dict(policy='step', step=[40, 80])\n",
      "total_epochs = 30\n",
      "checkpoint_config = dict(interval=10)\n",
      "evaluation = dict(\n",
      "    interval=5, metrics=['top_k_accuracy', 'mean_class_accuracy'])\n",
      "log_config = dict(interval=5, hooks=[dict(type='TextLoggerHook')])\n",
      "dist_params = dict(backend='nccl')\n",
      "log_level = 'INFO'\n",
      "work_dir = './tutorial_exps'\n",
      "load_from = './checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth'\n",
      "resume_from = None\n",
      "workflow = [('train', 1)]\n",
      "omnisource = False\n",
      "seed = 0\n",
      "gpu_ids = range(0, 1)\n",
      "\n"
     ],
     "name": "stdout"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tES-qnZ3k38Z"
   },
   "source": [
    "### Train a new recognizer\n",
    "\n",
    "Finally, lets initialize the dataset and recognizer, then train a new recognizer!"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "dDBWkdDRk6oz",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "85a52ef3-7b5c-4c52-8fef-00322a8c65e6"
   },
   "source": [
    "import os.path as osp\n",
    "\n",
    "from mmaction.datasets import build_dataset\n",
    "from mmaction.models import build_model\n",
    "from mmaction.apis import train_model\n",
    "\n",
    "import mmcv\n",
    "\n",
    "# Build the dataset\n",
    "datasets = [build_dataset(cfg.data.train)]\n",
    "\n",
    "# Build the recognizer\n",
    "model = build_model(cfg.model, train_cfg=cfg.get('train_cfg'), test_cfg=cfg.get('test_cfg'))\n",
    "\n",
    "# Create work_dir\n",
    "mmcv.mkdir_or_exist(osp.abspath(cfg.work_dir))\n",
    "train_model(model, datasets, cfg, distributed=False, validate=True)"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:38:54,909 - mmaction - INFO - These parameters in pretrained checkpoint are not loaded: {'fc.weight', 'fc.bias'}\n",
      "2020-11-20 09:38:54,960 - mmaction - INFO - load checkpoint from ./checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth\n",
      "2020-11-20 09:38:55,052 - mmaction - WARNING - The model and loaded state dict do not match exactly\n",
      "\n",
      "size mismatch for cls_head.fc_cls.weight: copying a param with shape torch.Size([400, 2048]) from checkpoint, the shape in current model is torch.Size([2, 2048]).\n",
      "size mismatch for cls_head.fc_cls.bias: copying a param with shape torch.Size([400]) from checkpoint, the shape in current model is torch.Size([2]).\n",
      "2020-11-20 09:38:55,056 - mmaction - INFO - Start running, host: root@74e02ee9f123, work_dir: /content/mmaction2/tutorial_exps\n",
      "2020-11-20 09:38:55,061 - mmaction - INFO - workflow: [('train', 1)], max: 30 epochs\n",
      "2020-11-20 09:38:59,709 - mmaction - INFO - Epoch [1][5/15]\tlr: 7.813e-05, eta: 0:06:52, time: 0.927, data_time: 0.708, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6865, loss: 0.6865, grad_norm: 12.7663\n",
      "2020-11-20 09:39:01,247 - mmaction - INFO - Epoch [1][10/15]\tlr: 7.813e-05, eta: 0:04:32, time: 0.309, data_time: 0.106, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.7171, loss: 0.7171, grad_norm: 13.7446\n",
      "2020-11-20 09:39:02,112 - mmaction - INFO - Epoch [1][15/15]\tlr: 7.813e-05, eta: 0:03:24, time: 0.173, data_time: 0.001, memory: 2918, top1_acc: 0.2000, top5_acc: 1.0000, loss_cls: 0.8884, loss: 0.8884, grad_norm: 14.7140\n",
      "2020-11-20 09:39:06,596 - mmaction - INFO - Epoch [2][5/15]\tlr: 7.813e-05, eta: 0:04:05, time: 0.876, data_time: 0.659, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.6562, loss: 0.6562, grad_norm: 10.5716\n",
      "2020-11-20 09:39:08,104 - mmaction - INFO - Epoch [2][10/15]\tlr: 7.813e-05, eta: 0:03:39, time: 0.300, data_time: 0.081, memory: 2918, top1_acc: 0.2000, top5_acc: 1.0000, loss_cls: 0.7480, loss: 0.7480, grad_norm: 11.7083\n",
      "2020-11-20 09:39:09,075 - mmaction - INFO - Epoch [2][15/15]\tlr: 7.813e-05, eta: 0:03:14, time: 0.195, data_time: 0.008, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6735, loss: 0.6735, grad_norm: 12.8046\n",
      "2020-11-20 09:39:13,756 - mmaction - INFO - Epoch [3][5/15]\tlr: 7.813e-05, eta: 0:03:39, time: 0.914, data_time: 0.693, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.7218, loss: 0.7218, grad_norm: 12.4893\n",
      "2020-11-20 09:39:15,203 - mmaction - INFO - Epoch [3][10/15]\tlr: 7.813e-05, eta: 0:03:24, time: 0.291, data_time: 0.092, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.6188, loss: 0.6188, grad_norm: 11.8106\n",
      "2020-11-20 09:39:16,108 - mmaction - INFO - Epoch [3][15/15]\tlr: 7.813e-05, eta: 0:03:07, time: 0.181, data_time: 0.003, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.7298, loss: 0.7298, grad_norm: 12.5043\n",
      "2020-11-20 09:39:21,525 - mmaction - INFO - Epoch [4][5/15]\tlr: 7.813e-05, eta: 0:03:29, time: 1.062, data_time: 0.832, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.6833, loss: 0.6833, grad_norm: 10.1046\n",
      "2020-11-20 09:39:22,815 - mmaction - INFO - Epoch [4][10/15]\tlr: 7.813e-05, eta: 0:03:17, time: 0.258, data_time: 0.059, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6640, loss: 0.6640, grad_norm: 11.7589\n",
      "2020-11-20 09:39:23,686 - mmaction - INFO - Epoch [4][15/15]\tlr: 7.813e-05, eta: 0:03:03, time: 0.174, data_time: 0.001, memory: 2918, top1_acc: 0.3000, top5_acc: 1.0000, loss_cls: 0.7372, loss: 0.7372, grad_norm: 13.6163\n",
      "2020-11-20 09:39:28,818 - mmaction - INFO - Epoch [5][5/15]\tlr: 7.813e-05, eta: 0:03:17, time: 1.001, data_time: 0.767, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.6309, loss: 0.6309, grad_norm: 11.1864\n",
      "2020-11-20 09:39:29,915 - mmaction - INFO - Epoch [5][10/15]\tlr: 7.813e-05, eta: 0:03:06, time: 0.220, data_time: 0.005, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.7178, loss: 0.7178, grad_norm: 12.4574\n",
      "2020-11-20 09:39:31,063 - mmaction - INFO - Epoch [5][15/15]\tlr: 7.813e-05, eta: 0:02:57, time: 0.229, data_time: 0.052, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.7094, loss: 0.7094, grad_norm: 12.4649\n"
     ],
     "name": "stderr"
    },
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 4.0 task/s, elapsed: 3s, ETA:     0s"
     ],
     "name": "stdout"
    },
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:39:33,791 - mmaction - INFO - Evaluating top_k_accuracy ...\n",
      "2020-11-20 09:39:33,793 - mmaction - INFO - \n",
      "top1_acc\t0.7000\n",
      "top5_acc\t1.0000\n",
      "2020-11-20 09:39:33,794 - mmaction - INFO - Evaluating mean_class_accuracy ...\n",
      "2020-11-20 09:39:33,797 - mmaction - INFO - \n",
      "mean_acc\t0.7000\n",
      "2020-11-20 09:39:33,798 - mmaction - INFO - Now best checkpoint is epoch_5.pth\n",
      "2020-11-20 09:39:33,801 - mmaction - INFO - Epoch(val) [5][15]\ttop1_acc: 0.7000, top5_acc: 1.0000, mean_class_accuracy: 0.7000\n",
      "2020-11-20 09:39:39,131 - mmaction - INFO - Epoch [6][5/15]\tlr: 7.813e-05, eta: 0:03:09, time: 1.065, data_time: 0.832, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.6696, loss: 0.6696, grad_norm: 11.0206\n",
      "2020-11-20 09:39:40,263 - mmaction - INFO - Epoch [6][10/15]\tlr: 7.813e-05, eta: 0:03:00, time: 0.226, data_time: 0.002, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6824, loss: 0.6824, grad_norm: 11.8897\n",
      "2020-11-20 09:39:41,147 - mmaction - INFO - Epoch [6][15/15]\tlr: 7.813e-05, eta: 0:02:51, time: 0.177, data_time: 0.002, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6869, loss: 0.6869, grad_norm: 13.7294\n",
      "2020-11-20 09:39:45,927 - mmaction - INFO - Epoch [7][5/15]\tlr: 7.813e-05, eta: 0:02:57, time: 0.934, data_time: 0.712, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6459, loss: 0.6459, grad_norm: 10.8083\n",
      "2020-11-20 09:39:47,203 - mmaction - INFO - Epoch [7][10/15]\tlr: 7.813e-05, eta: 0:02:50, time: 0.255, data_time: 0.038, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.6004, loss: 0.6004, grad_norm: 9.5414\n",
      "2020-11-20 09:39:48,386 - mmaction - INFO - Epoch [7][15/15]\tlr: 7.813e-05, eta: 0:02:44, time: 0.237, data_time: 0.056, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6829, loss: 0.6829, grad_norm: 12.0934\n",
      "2020-11-20 09:39:53,096 - mmaction - INFO - Epoch [8][5/15]\tlr: 7.813e-05, eta: 0:02:48, time: 0.920, data_time: 0.701, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.6921, loss: 0.6921, grad_norm: 13.1963\n",
      "2020-11-20 09:39:54,671 - mmaction - INFO - Epoch [8][10/15]\tlr: 7.813e-05, eta: 0:02:43, time: 0.314, data_time: 0.114, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6073, loss: 0.6073, grad_norm: 11.3379\n",
      "2020-11-20 09:39:55,578 - mmaction - INFO - Epoch [8][15/15]\tlr: 7.813e-05, eta: 0:02:37, time: 0.183, data_time: 0.002, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.7029, loss: 0.7029, grad_norm: 12.4497\n",
      "2020-11-20 09:40:00,149 - mmaction - INFO - Epoch [9][5/15]\tlr: 7.813e-05, eta: 0:02:40, time: 0.893, data_time: 0.662, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.5367, loss: 0.5367, grad_norm: 8.4173\n",
      "2020-11-20 09:40:01,830 - mmaction - INFO - Epoch [9][10/15]\tlr: 7.813e-05, eta: 0:02:35, time: 0.336, data_time: 0.127, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6797, loss: 0.6797, grad_norm: 12.8048\n",
      "2020-11-20 09:40:02,743 - mmaction - INFO - Epoch [9][15/15]\tlr: 7.813e-05, eta: 0:02:29, time: 0.182, data_time: 0.001, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5711, loss: 0.5711, grad_norm: 9.9660\n",
      "2020-11-20 09:40:07,240 - mmaction - INFO - Epoch [10][5/15]\tlr: 7.813e-05, eta: 0:02:31, time: 0.878, data_time: 0.652, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5379, loss: 0.5379, grad_norm: 8.8850\n",
      "2020-11-20 09:40:08,703 - mmaction - INFO - Epoch [10][10/15]\tlr: 7.813e-05, eta: 0:02:27, time: 0.293, data_time: 0.075, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.6644, loss: 0.6644, grad_norm: 12.6150\n",
      "2020-11-20 09:40:09,886 - mmaction - INFO - Epoch [10][15/15]\tlr: 7.813e-05, eta: 0:02:22, time: 0.236, data_time: 0.047, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.7041, loss: 0.7041, grad_norm: 12.4941\n",
      "2020-11-20 09:40:09,990 - mmaction - INFO - Saving checkpoint at 10 epochs\n"
     ],
     "name": "stderr"
    },
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 4.2 task/s, elapsed: 2s, ETA:     0s"
     ],
     "name": "stdout"
    },
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:40:12,776 - mmaction - INFO - Evaluating top_k_accuracy ...\n",
      "2020-11-20 09:40:12,778 - mmaction - INFO - \n",
      "top1_acc\t0.9000\n",
      "top5_acc\t1.0000\n",
      "2020-11-20 09:40:12,778 - mmaction - INFO - Evaluating mean_class_accuracy ...\n",
      "2020-11-20 09:40:12,782 - mmaction - INFO - \n",
      "mean_acc\t0.9000\n",
      "2020-11-20 09:40:12,783 - mmaction - INFO - Now best checkpoint is epoch_10.pth\n",
      "2020-11-20 09:40:12,786 - mmaction - INFO - Epoch(val) [10][15]\ttop1_acc: 0.9000, top5_acc: 1.0000, mean_class_accuracy: 0.9000\n",
      "2020-11-20 09:40:17,313 - mmaction - INFO - Epoch [11][5/15]\tlr: 7.813e-05, eta: 0:02:24, time: 0.904, data_time: 0.671, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5357, loss: 0.5357, grad_norm: 8.7986\n",
      "2020-11-20 09:40:18,877 - mmaction - INFO - Epoch [11][10/15]\tlr: 7.813e-05, eta: 0:02:20, time: 0.313, data_time: 0.097, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5829, loss: 0.5829, grad_norm: 10.5930\n",
      "2020-11-20 09:40:19,797 - mmaction - INFO - Epoch [11][15/15]\tlr: 7.813e-05, eta: 0:02:15, time: 0.184, data_time: 0.003, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6232, loss: 0.6232, grad_norm: 11.3676\n",
      "2020-11-20 09:40:24,593 - mmaction - INFO - Epoch [12][5/15]\tlr: 7.813e-05, eta: 0:02:16, time: 0.936, data_time: 0.689, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5701, loss: 0.5701, grad_norm: 10.0657\n",
      "2020-11-20 09:40:25,850 - mmaction - INFO - Epoch [12][10/15]\tlr: 7.813e-05, eta: 0:02:12, time: 0.253, data_time: 0.045, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.5980, loss: 0.5980, grad_norm: 11.1706\n",
      "2020-11-20 09:40:26,805 - mmaction - INFO - Epoch [12][15/15]\tlr: 7.813e-05, eta: 0:02:07, time: 0.191, data_time: 0.002, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.6105, loss: 0.6105, grad_norm: 11.8521\n",
      "2020-11-20 09:40:31,590 - mmaction - INFO - Epoch [13][5/15]\tlr: 7.813e-05, eta: 0:02:08, time: 0.935, data_time: 0.712, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5439, loss: 0.5439, grad_norm: 10.4133\n",
      "2020-11-20 09:40:32,717 - mmaction - INFO - Epoch [13][10/15]\tlr: 7.813e-05, eta: 0:02:04, time: 0.225, data_time: 0.002, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6338, loss: 0.6338, grad_norm: 11.3842\n",
      "2020-11-20 09:40:33,827 - mmaction - INFO - Epoch [13][15/15]\tlr: 7.813e-05, eta: 0:02:00, time: 0.222, data_time: 0.036, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6328, loss: 0.6328, grad_norm: 12.0624\n",
      "2020-11-20 09:40:38,599 - mmaction - INFO - Epoch [14][5/15]\tlr: 7.813e-05, eta: 0:02:00, time: 0.932, data_time: 0.695, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6514, loss: 0.6514, grad_norm: 12.4955\n",
      "2020-11-20 09:40:40,073 - mmaction - INFO - Epoch [14][10/15]\tlr: 7.813e-05, eta: 0:01:57, time: 0.295, data_time: 0.085, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5536, loss: 0.5536, grad_norm: 10.5949\n",
      "2020-11-20 09:40:40,980 - mmaction - INFO - Epoch [14][15/15]\tlr: 7.813e-05, eta: 0:01:53, time: 0.181, data_time: 0.001, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.7072, loss: 0.7072, grad_norm: 12.8191\n",
      "2020-11-20 09:40:46,296 - mmaction - INFO - Epoch [15][5/15]\tlr: 7.813e-05, eta: 0:01:53, time: 1.042, data_time: 0.819, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6580, loss: 0.6580, grad_norm: 12.7106\n",
      "2020-11-20 09:40:47,307 - mmaction - INFO - Epoch [15][10/15]\tlr: 7.813e-05, eta: 0:01:50, time: 0.202, data_time: 0.001, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.4524, loss: 0.4524, grad_norm: 8.1145\n",
      "2020-11-20 09:40:48,205 - mmaction - INFO - Epoch [15][15/15]\tlr: 7.813e-05, eta: 0:01:46, time: 0.179, data_time: 0.002, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6318, loss: 0.6318, grad_norm: 11.7249\n"
     ],
     "name": "stderr"
    },
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 4.3 task/s, elapsed: 2s, ETA:     0s"
     ],
     "name": "stdout"
    },
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:40:50,731 - mmaction - INFO - Evaluating top_k_accuracy ...\n",
      "2020-11-20 09:40:50,732 - mmaction - INFO - \n",
      "top1_acc\t0.8000\n",
      "top5_acc\t1.0000\n",
      "2020-11-20 09:40:50,733 - mmaction - INFO - Evaluating mean_class_accuracy ...\n",
      "2020-11-20 09:40:50,739 - mmaction - INFO - \n",
      "mean_acc\t0.8000\n",
      "2020-11-20 09:40:50,741 - mmaction - INFO - Epoch(val) [15][15]\ttop1_acc: 0.8000, top5_acc: 1.0000, mean_class_accuracy: 0.8000\n",
      "2020-11-20 09:40:55,222 - mmaction - INFO - Epoch [16][5/15]\tlr: 7.813e-05, eta: 0:01:45, time: 0.895, data_time: 0.658, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5428, loss: 0.5428, grad_norm: 10.0143\n",
      "2020-11-20 09:40:56,773 - mmaction - INFO - Epoch [16][10/15]\tlr: 7.813e-05, eta: 0:01:42, time: 0.310, data_time: 0.102, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6099, loss: 0.6099, grad_norm: 11.1996\n",
      "2020-11-20 09:40:57,703 - mmaction - INFO - Epoch [16][15/15]\tlr: 7.813e-05, eta: 0:01:38, time: 0.186, data_time: 0.002, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5309, loss: 0.5309, grad_norm: 9.9062\n",
      "2020-11-20 09:41:02,293 - mmaction - INFO - Epoch [17][5/15]\tlr: 7.813e-05, eta: 0:01:38, time: 0.897, data_time: 0.665, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6303, loss: 0.6303, grad_norm: 12.2689\n",
      "2020-11-20 09:41:03,790 - mmaction - INFO - Epoch [17][10/15]\tlr: 7.813e-05, eta: 0:01:35, time: 0.299, data_time: 0.094, memory: 2918, top1_acc: 0.4000, top5_acc: 1.0000, loss_cls: 0.6635, loss: 0.6635, grad_norm: 13.0193\n",
      "2020-11-20 09:41:04,739 - mmaction - INFO - Epoch [17][15/15]\tlr: 7.813e-05, eta: 0:01:31, time: 0.190, data_time: 0.011, memory: 2918, top1_acc: 1.0000, top5_acc: 1.0000, loss_cls: 0.4261, loss: 0.4261, grad_norm: 8.0281\n",
      "2020-11-20 09:41:09,499 - mmaction - INFO - Epoch [18][5/15]\tlr: 7.813e-05, eta: 0:01:31, time: 0.929, data_time: 0.694, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5793, loss: 0.5793, grad_norm: 11.0233\n",
      "2020-11-20 09:41:11,013 - mmaction - INFO - Epoch [18][10/15]\tlr: 7.813e-05, eta: 0:01:28, time: 0.303, data_time: 0.089, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.4537, loss: 0.4537, grad_norm: 8.3300\n",
      "2020-11-20 09:41:11,913 - mmaction - INFO - Epoch [18][15/15]\tlr: 7.813e-05, eta: 0:01:24, time: 0.180, data_time: 0.002, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4951, loss: 0.4951, grad_norm: 9.5771\n",
      "2020-11-20 09:41:16,921 - mmaction - INFO - Epoch [19][5/15]\tlr: 7.813e-05, eta: 0:01:24, time: 0.979, data_time: 0.743, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.5532, loss: 0.5532, grad_norm: 10.2661\n",
      "2020-11-20 09:41:18,235 - mmaction - INFO - Epoch [19][10/15]\tlr: 7.813e-05, eta: 0:01:20, time: 0.263, data_time: 0.059, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6117, loss: 0.6117, grad_norm: 11.5430\n",
      "2020-11-20 09:41:19,137 - mmaction - INFO - Epoch [19][15/15]\tlr: 7.813e-05, eta: 0:01:17, time: 0.180, data_time: 0.001, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4127, loss: 0.4127, grad_norm: 7.5687\n",
      "2020-11-20 09:41:23,780 - mmaction - INFO - Epoch [20][5/15]\tlr: 7.813e-05, eta: 0:01:16, time: 0.904, data_time: 0.675, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.4327, loss: 0.4327, grad_norm: 8.4868\n",
      "2020-11-20 09:41:25,236 - mmaction - INFO - Epoch [20][10/15]\tlr: 7.813e-05, eta: 0:01:13, time: 0.293, data_time: 0.078, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4623, loss: 0.4623, grad_norm: 8.9795\n",
      "2020-11-20 09:41:26,198 - mmaction - INFO - Epoch [20][15/15]\tlr: 7.813e-05, eta: 0:01:10, time: 0.194, data_time: 0.002, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5917, loss: 0.5917, grad_norm: 11.3897\n",
      "2020-11-20 09:41:26,304 - mmaction - INFO - Saving checkpoint at 20 epochs\n"
     ],
     "name": "stderr"
    },
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 4.3 task/s, elapsed: 2s, ETA:     0s"
     ],
     "name": "stdout"
    },
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:41:29,078 - mmaction - INFO - Evaluating top_k_accuracy ...\n",
      "2020-11-20 09:41:29,079 - mmaction - INFO - \n",
      "top1_acc\t0.9000\n",
      "top5_acc\t1.0000\n",
      "2020-11-20 09:41:29,080 - mmaction - INFO - Evaluating mean_class_accuracy ...\n",
      "2020-11-20 09:41:29,084 - mmaction - INFO - \n",
      "mean_acc\t0.9000\n",
      "2020-11-20 09:41:29,086 - mmaction - INFO - Epoch(val) [20][15]\ttop1_acc: 0.9000, top5_acc: 1.0000, mean_class_accuracy: 0.9000\n",
      "2020-11-20 09:41:33,520 - mmaction - INFO - Epoch [21][5/15]\tlr: 7.813e-05, eta: 0:01:09, time: 0.886, data_time: 0.653, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4515, loss: 0.4515, grad_norm: 8.3883\n",
      "2020-11-20 09:41:35,097 - mmaction - INFO - Epoch [21][10/15]\tlr: 7.813e-05, eta: 0:01:06, time: 0.316, data_time: 0.095, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4543, loss: 0.4543, grad_norm: 9.1107\n",
      "2020-11-20 09:41:36,032 - mmaction - INFO - Epoch [21][15/15]\tlr: 7.813e-05, eta: 0:01:03, time: 0.187, data_time: 0.001, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5324, loss: 0.5324, grad_norm: 10.5992\n",
      "2020-11-20 09:41:40,665 - mmaction - INFO - Epoch [22][5/15]\tlr: 7.813e-05, eta: 0:01:02, time: 0.905, data_time: 0.671, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5645, loss: 0.5645, grad_norm: 10.7946\n",
      "2020-11-20 09:41:42,016 - mmaction - INFO - Epoch [22][10/15]\tlr: 7.813e-05, eta: 0:00:59, time: 0.270, data_time: 0.061, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.5059, loss: 0.5059, grad_norm: 9.9750\n",
      "2020-11-20 09:41:43,160 - mmaction - INFO - Epoch [22][15/15]\tlr: 7.813e-05, eta: 0:00:56, time: 0.229, data_time: 0.049, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.4390, loss: 0.4390, grad_norm: 8.7574\n",
      "2020-11-20 09:41:47,850 - mmaction - INFO - Epoch [23][5/15]\tlr: 7.813e-05, eta: 0:00:54, time: 0.916, data_time: 0.686, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.6279, loss: 0.6279, grad_norm: 12.5599\n",
      "2020-11-20 09:41:49,318 - mmaction - INFO - Epoch [23][10/15]\tlr: 7.813e-05, eta: 0:00:52, time: 0.293, data_time: 0.080, memory: 2918, top1_acc: 1.0000, top5_acc: 1.0000, loss_cls: 0.3282, loss: 0.3282, grad_norm: 6.4802\n",
      "2020-11-20 09:41:50,252 - mmaction - INFO - Epoch [23][15/15]\tlr: 7.813e-05, eta: 0:00:49, time: 0.188, data_time: 0.003, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4970, loss: 0.4970, grad_norm: 9.2502\n",
      "2020-11-20 09:41:55,208 - mmaction - INFO - Epoch [24][5/15]\tlr: 7.813e-05, eta: 0:00:47, time: 0.970, data_time: 0.731, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.3843, loss: 0.3843, grad_norm: 7.6723\n",
      "2020-11-20 09:41:56,257 - mmaction - INFO - Epoch [24][10/15]\tlr: 7.813e-05, eta: 0:00:44, time: 0.210, data_time: 0.006, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.5481, loss: 0.5481, grad_norm: 11.8595\n",
      "2020-11-20 09:41:57,251 - mmaction - INFO - Epoch [24][15/15]\tlr: 7.813e-05, eta: 0:00:42, time: 0.199, data_time: 0.012, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.3737, loss: 0.3737, grad_norm: 7.4704\n",
      "2020-11-20 09:42:01,669 - mmaction - INFO - Epoch [25][5/15]\tlr: 7.813e-05, eta: 0:00:40, time: 0.863, data_time: 0.635, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.5808, loss: 0.5808, grad_norm: 11.7501\n",
      "2020-11-20 09:42:03,132 - mmaction - INFO - Epoch [25][10/15]\tlr: 7.813e-05, eta: 0:00:37, time: 0.292, data_time: 0.075, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.5970, loss: 0.5970, grad_norm: 11.7784\n",
      "2020-11-20 09:42:04,287 - mmaction - INFO - Epoch [25][15/15]\tlr: 7.813e-05, eta: 0:00:35, time: 0.232, data_time: 0.042, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4509, loss: 0.4509, grad_norm: 9.2979\n"
     ],
     "name": "stderr"
    },
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 4.4 task/s, elapsed: 2s, ETA:     0s"
     ],
     "name": "stdout"
    },
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:42:06,788 - mmaction - INFO - Evaluating top_k_accuracy ...\n",
      "2020-11-20 09:42:06,790 - mmaction - INFO - \n",
      "top1_acc\t1.0000\n",
      "top5_acc\t1.0000\n",
      "2020-11-20 09:42:06,791 - mmaction - INFO - Evaluating mean_class_accuracy ...\n",
      "2020-11-20 09:42:06,796 - mmaction - INFO - \n",
      "mean_acc\t1.0000\n",
      "2020-11-20 09:42:06,797 - mmaction - INFO - Now best checkpoint is epoch_25.pth\n",
      "2020-11-20 09:42:06,799 - mmaction - INFO - Epoch(val) [25][15]\ttop1_acc: 1.0000, top5_acc: 1.0000, mean_class_accuracy: 1.0000\n",
      "2020-11-20 09:42:11,579 - mmaction - INFO - Epoch [26][5/15]\tlr: 7.813e-05, eta: 0:00:33, time: 0.954, data_time: 0.725, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.3759, loss: 0.3759, grad_norm: 7.9644\n",
      "2020-11-20 09:42:13,112 - mmaction - INFO - Epoch [26][10/15]\tlr: 7.813e-05, eta: 0:00:30, time: 0.307, data_time: 0.114, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.5393, loss: 0.5393, grad_norm: 10.7580\n",
      "2020-11-20 09:42:14,007 - mmaction - INFO - Epoch [26][15/15]\tlr: 7.813e-05, eta: 0:00:28, time: 0.179, data_time: 0.002, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.4193, loss: 0.4193, grad_norm: 8.5498\n",
      "2020-11-20 09:42:18,684 - mmaction - INFO - Epoch [27][5/15]\tlr: 7.813e-05, eta: 0:00:26, time: 0.914, data_time: 0.690, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.5922, loss: 0.5922, grad_norm: 11.3893\n",
      "2020-11-20 09:42:19,933 - mmaction - INFO - Epoch [27][10/15]\tlr: 7.813e-05, eta: 0:00:23, time: 0.249, data_time: 0.031, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.4481, loss: 0.4481, grad_norm: 9.5221\n",
      "2020-11-20 09:42:21,051 - mmaction - INFO - Epoch [27][15/15]\tlr: 7.813e-05, eta: 0:00:21, time: 0.224, data_time: 0.040, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.3864, loss: 0.3864, grad_norm: 8.2003\n",
      "2020-11-20 09:42:25,678 - mmaction - INFO - Epoch [28][5/15]\tlr: 7.813e-05, eta: 0:00:18, time: 0.904, data_time: 0.668, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5439, loss: 0.5439, grad_norm: 11.1966\n",
      "2020-11-20 09:42:26,792 - mmaction - INFO - Epoch [28][10/15]\tlr: 7.813e-05, eta: 0:00:16, time: 0.221, data_time: 0.003, memory: 2918, top1_acc: 0.8000, top5_acc: 1.0000, loss_cls: 0.4840, loss: 0.4840, grad_norm: 9.9172\n",
      "2020-11-20 09:42:27,960 - mmaction - INFO - Epoch [28][15/15]\tlr: 7.813e-05, eta: 0:00:14, time: 0.235, data_time: 0.044, memory: 2918, top1_acc: 0.9000, top5_acc: 1.0000, loss_cls: 0.3131, loss: 0.3131, grad_norm: 6.3335\n",
      "2020-11-20 09:42:32,811 - mmaction - INFO - Epoch [29][5/15]\tlr: 7.813e-05, eta: 0:00:11, time: 0.948, data_time: 0.722, memory: 2918, top1_acc: 1.0000, top5_acc: 1.0000, loss_cls: 0.3146, loss: 0.3146, grad_norm: 6.6830\n",
      "2020-11-20 09:42:34,156 - mmaction - INFO - Epoch [29][10/15]\tlr: 7.813e-05, eta: 0:00:09, time: 0.269, data_time: 0.062, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.4965, loss: 0.4965, grad_norm: 10.3011\n",
      "2020-11-20 09:42:35,103 - mmaction - INFO - Epoch [29][15/15]\tlr: 7.813e-05, eta: 0:00:07, time: 0.189, data_time: 0.013, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.5301, loss: 0.5301, grad_norm: 10.5820\n",
      "2020-11-20 09:42:39,945 - mmaction - INFO - Epoch [30][5/15]\tlr: 7.813e-05, eta: 0:00:04, time: 0.945, data_time: 0.720, memory: 2918, top1_acc: 0.5000, top5_acc: 1.0000, loss_cls: 0.6774, loss: 0.6774, grad_norm: 13.5632\n",
      "2020-11-20 09:42:41,213 - mmaction - INFO - Epoch [30][10/15]\tlr: 7.813e-05, eta: 0:00:02, time: 0.255, data_time: 0.037, memory: 2918, top1_acc: 0.7000, top5_acc: 1.0000, loss_cls: 0.6271, loss: 0.6271, grad_norm: 12.5629\n",
      "2020-11-20 09:42:42,162 - mmaction - INFO - Epoch [30][15/15]\tlr: 7.813e-05, eta: 0:00:00, time: 0.190, data_time: 0.006, memory: 2918, top1_acc: 0.6000, top5_acc: 1.0000, loss_cls: 0.5681, loss: 0.5681, grad_norm: 11.7026\n",
      "2020-11-20 09:42:42,265 - mmaction - INFO - Saving checkpoint at 30 epochs\n"
     ],
     "name": "stderr"
    },
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 4.2 task/s, elapsed: 2s, ETA:     0s"
     ],
     "name": "stdout"
    },
    {
     "output_type": "stream",
     "text": [
      "2020-11-20 09:42:45,097 - mmaction - INFO - Evaluating top_k_accuracy ...\n",
      "2020-11-20 09:42:45,098 - mmaction - INFO - \n",
      "top1_acc\t1.0000\n",
      "top5_acc\t1.0000\n",
      "2020-11-20 09:42:45,099 - mmaction - INFO - Evaluating mean_class_accuracy ...\n",
      "2020-11-20 09:42:45,103 - mmaction - INFO - \n",
      "mean_acc\t1.0000\n",
      "2020-11-20 09:42:45,104 - mmaction - INFO - Epoch(val) [30][15]\ttop1_acc: 1.0000, top5_acc: 1.0000, mean_class_accuracy: 1.0000\n"
     ],
     "name": "stderr"
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zdSd7oTLlxIf"
   },
   "source": [
    "### Understand the log\n",
    "From the log, we can have a basic understanding the training process and know how well the recognizer is trained.\n",
    "\n",
    "Firstly, the ResNet-50 backbone pre-trained on ImageNet is loaded, this is a common practice since training from scratch is more cost. The log shows that all the weights of the ResNet-50 backbone are loaded except the `fc.bias` and `fc.weight`.\n",
    "\n",
    "Second, since the dataset we are using is small, we loaded a TSN model and finetune it for action recognition.\n",
    "The original TSN is trained on original Kinetics-400 dataset which contains 400 classes but Kinetics-400 Tiny dataset only have 2 classes. Therefore, the last FC layer of the pre-trained TSN for classification has different weight shape and is not used.\n",
    "\n",
    "Third, after training, the recognizer is evaluated by the default evaluation. The results show that the recognizer achieves 100% top1 accuracy and 100% top5 accuracy on the val dataset,\n",
    " \n",
    "Not bad!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ryVoSfZVmogw"
   },
   "source": [
    "## Test the trained recognizer\n",
    "\n",
    "After finetuning the recognizer, let's check the prediction results!"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "id": "eyY3hCMwyTct",
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "outputId": "200e37c7-0da4-421f-98da-41418c3110ca"
   },
   "source": [
    "from mmaction.apis import single_gpu_test\n",
    "from mmaction.datasets import build_dataloader\n",
    "from mmcv.parallel import MMDataParallel\n",
    "\n",
    "# Build a test dataloader\n",
    "dataset = build_dataset(cfg.data.test, dict(test_mode=True))\n",
    "data_loader = build_dataloader(\n",
    "        dataset,\n",
    "        videos_per_gpu=1,\n",
    "        workers_per_gpu=cfg.data.workers_per_gpu,\n",
    "        dist=False,\n",
    "        shuffle=False)\n",
    "model = MMDataParallel(model, device_ids=[0])\n",
    "outputs = single_gpu_test(model, data_loader)\n",
    "\n",
    "eval_config = cfg.evaluation\n",
    "eval_config.pop('interval')\n",
    "eval_res = dataset.evaluate(outputs, **eval_config)\n",
    "for name, val in eval_res.items():\n",
    "    print(f'{name}: {val:.04f}')"
   ],
   "execution_count": null,
   "outputs": [
    {
     "output_type": "stream",
     "text": [
      "[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10/10, 1.9 task/s, elapsed: 5s, ETA:     0s\n",
      "Evaluating top_k_accuracy ...\n",
      "\n",
      "top1_acc\t1.0000\n",
      "top5_acc\t1.0000\n",
      "\n",
      "Evaluating mean_class_accuracy ...\n",
      "\n",
      "mean_acc\t1.0000\n",
      "top1_acc: 1.0000\n",
      "top5_acc: 1.0000\n",
      "mean_class_accuracy: 1.0000\n"
     ],
     "name": "stdout"
    }
   ]
  }
 ]
}
